A cost-benefit analysis of low latency via added utilization

Many techniques for improving latency in the Internet trade trade off some amount of extra bandwidth consumption for reduced latency. Examples include DNS prefetching [2] and redundancy [5]. But when is the latency reduction achieved worth the cost of the added overhead? In this brief note, we develop a benchmark for answering this question via an economic cost vs. benefit analysis. We consider the tradeoff between cost and benefit in a specific class of systems: wide-area client-server applications (such as web browsing, DNS queries, etc.) involving clients using consumer-level connectivity and service providers in the cloud. The framework we develop here serves as a baseline; it can be refined or extended for other systems. To summarize the benchmark in one sentence: any technique that saves more than 16 ms of latency (in the mean or the tail, depending on the metric we are concerned with) for every kilobyte of extra traffic that it sends is useful, even with very pessimistic estimates for the additional cost induced at both clients and servers. We estimate when a given technique is useful via the value of time and the cost of increased utilization. We will denote increased utilization in units of data transfer volume. Note however that our calculated costs include all costs incurred by that added utilization, not just bandwidth. For example, one kilobyte of added client-side traffic in a web service has server-side costs including server utilization, energy, network operations staff, and so on. In other words, these diverse costs are amortized over units of client-side traffic. For the latency reduction to be beneficial, we need lv ≥ c, where l is the average latency savings in milliseconds for each KB of added traffic, v is the dollar value of one millisecond of latency reduction, and c is the cost of added traffic per KB. Our goal is to find a threshold value of l when the latency improvement becomes beneficial. For this, we need estimates of c and v. These are very different on the client and server sides.